For a fistful of dollars: using crowd-sourcing to evaluate a spoken language CALL application
نویسندگان
چکیده
We present an evaluation of a Web-deployed spoken language CALL system, carried out using crowd-sourcing methods. The system, “Survival Japanese”, is a crash course in tourist Japanese implemented within the platform CALL-SLT. The evaluation was carried out over one week using the Amazon Mechanical Turk. Although we found a high proportion of attempted scammers, there was a core of 23 subjects who used the system in a responsible manner. The evidence that these subjects learned from their 111 sessions and 9092 spoken interactions was significant at P = 0.001. Our conclusion is that crowd-sourcing is a potentially valid method for evaluating spoken CALL systems.
منابع مشابه
Multimodal Corpus Using Multimodal Dictionary in Lohorung
Lohorung is one of the minority Tibeto-Burman languages of Kirati group spoken in the Eastern part of Nepal. It lacks a written tradition. Crowd sourcing can be a good way to document a language in terms of resources, especially in the context of a linguistically rich country like Nepal. We have built a multimodal dictionary, browsing and authoring tool with which Lohorung community members, ev...
متن کاملMining Search Query Logs for Spoken Language Understanding
In a spoken dialog system that can handle natural conversation between a human and a machine, spoken language understanding (SLU) is a crucial component aiming at capturing the key semantic components of utterances. Building a robust SLU system is a challenging task due to variability in the usage of language, need for labeled data, and requirements to expand to new domains (movies, travel, fin...
متن کاملA Conversational Movie Search System Based on Conditional Random Fields
Online streaming companies such as Netflix have become dominant in the media distribution sector. However, such media delivery services often support very rudimentary search, especially for natural language queries. To provide a more natural search interface, we have developed a conversational movie search system, which parses the recognition hypothesis of a spoken query into semantic classes u...
متن کاملActive Learning and Crowd-Sourcing for Machine Translation
In recent years, corpus based approaches to machine translation have become predominant, with Statistical Machine Translation (SMT) being the most actively progressing area. Success of these approaches depends on the availability of parallel corpora. In this paper we propose Active Crowd Translation (ACT), a new paradigm where active learning and crowd-sourcing come together to enable automatic...
متن کاملTeaching approaches to Computer Assisted Language Learning
Computers have been used for language teaching ever since the 1960's.Learning a second language is a challenging endeavor, and, for decades now, proponents of computer assisted language learning (CALL) have declared that help is on the horison. We investigate the suitability of deploying speech technology in computer based systems that can be used to teach foreign language skills. In this case,...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2011